All contributors have signed the CLA. Thank you! ✅
_{Posted by the CLA Assistant Lite bot.}

I have read the CLA Document and I hereby sign the CLA

Greptile Summary

Adds Phase 0.5 Documentation Discovery workflow to improve documentation lookup accuracy before TYPE A (Conceptual) and TYPE D (Comprehensive) requests. The new workflow discovers official docs URL, verifies versioned documentation, fetches sitemap.xml to understand doc structure, then performs targeted investigation instead of random searching.

• Sequential Doc Discovery: websearch → version check → sitemap → targeted investigation
• Main Phase Remains Parallel: Once doc structure is known, parallel tool execution continues
• Classification table updated: Shows "Doc Discovery →" for TYPE A and D
• New tool entries: Added sitemap discovery and doc page fetching with webfetch
• Failure recovery: Handles sitemap/versioned docs not found cases

Issue Found: Changed tool name from context7_get-library-docs to context7_query-docs in three locations (lines 114, 181, 237), but src/hooks/agent-usage-reminder/constants.ts:18 still references the old tool name. This creates an inconsistency where the agent usage reminder hook won't recognize the tool being used.

Confidence Score: 3/5

• This PR is safe to merge after fixing the tool name inconsistency
• The workflow logic and documentation improvements are sound, but the tool name change from context7_get-library-docs to context7_query-docs creates a runtime inconsistency with the agent-usage-reminder hook that still references the old name. This must be corrected either by reverting the tool name changes or updating the hook configuration.
• Check src/hooks/agent-usage-reminder/constants.ts to ensure tool name matches the changes in src/agents/librarian.ts

Important Files Changed

Sequence Diagram

sequenceDiagram
    participant User
    participant Librarian
    participant WebSearch as websearch_exa
    participant Sitemap as webfetch(sitemap)
    participant Docs as webfetch(doc_pages)
    participant Context7
    participant GrepApp as grep_app
    
    Note over User,GrepApp: TYPE A or TYPE D Request
    
    User->>Librarian: "How do I use React 18?"
    
    rect rgb(230, 240, 255)
        Note right of Librarian: Phase 0.5: Documentation Discovery (SEQUENTIAL)
        Librarian->>WebSearch: "React official documentation site"
        WebSearch-->>Librarian: https://react.dev
        
        Librarian->>WebSearch: "React v18 documentation"
        WebSearch-->>Librarian: Versioned URL confirmed
        
        Librarian->>Sitemap: GET /sitemap.xml
        Sitemap-->>Librarian: Parse doc structure
        
        Note right of Librarian: Identify relevant sections from sitemap
    end
    
    rect rgb(240, 255, 240)
        Note right of Librarian: Phase 1: Main Investigation (PARALLEL)
        par Parallel Execution
            Librarian->>Context7: resolve-library-id("react")
            Context7-->>Librarian: library_id
            Librarian->>Context7: query-docs(id, "hooks")
            Context7-->>Librarian: Official docs
        and
            Librarian->>Docs: GET /docs/hooks.html (from sitemap)
            Docs-->>Librarian: Targeted doc page
        and
            Librarian->>GrepApp: searchGitHub("React hooks usage")
            GrepApp-->>Librarian: Code examples
        end
    end
    
    Librarian->>User: Synthesized answer with permalinks

@sisyphus-dev-ai review this

And how do you think this will change the librarian's behavior be like? Will it slower? Preciser? More possiblities to answer deprecated stuff?

ulw to share your opinion

Review: Documentation Discovery Workflow for Librarian

Great question @code-yeongyu! Let me break down the behavioral impact of this change.

TL;DR

Detailed Analysis

Speed Trade-off

Before: Immediate parallel fire (3+ calls for TYPE A, 6+ for TYPE D)

After: Sequential discovery → then parallel fire

websearch (find docs URL)
   ↓
webfetch (version check if specified)
   ↓
webfetch (sitemap.xml)
   ↓
[NOW parallel phase begins]

This adds ~2-4 seconds latency on TYPE A/D requests. The main investigation phase remains parallel.

Precision Improvement (The Real Win)

The current librarian does "hopeful parallel searching" - it fires multiple tools simultaneously without knowing WHERE the relevant documentation lives. This leads to:

• Random page hits that miss relevant content
• No understanding of doc organization
• Luck-dependent results

The new flow:

1. Finds official docs URL - not blogs, not tutorials
2. Verifies version - React 18 docs, not React 17
3. Parses sitemap - understands the doc structure
4. Fetches targeted pages - informed by sitemap knowledge

This is like the difference between grepping blindly vs. understanding the codebase structure first.

Deprecated Content (Counter-intuitive!)

You asked if this increases deprecated content risk. It's actually the opposite.

The new Phase 0.5 explicitly:

• Checks for versioned documentation when user specifies a version
• Falls back to latest with explicit acknowledgment: "Versioned docs not found → Fall back to latest version, note this in response"

Before this change, the librarian had zero version awareness. It would happily return Next.js 13 App Router docs when the user was asking about Next.js 14.

Trade-off Summary

Verdict

This is a good trade-off. Documentation questions benefit more from precision than raw speed. Users asking "How do I use X in version Y?" care about getting the right answer, not getting any answer 2 seconds faster.

The sequential discovery is a one-time upfront cost that dramatically improves the quality of the parallel phase that follows.

Minor Suggestion

Consider adding a skip condition for well-known libraries with consistent doc structures (React, Next.js, etc.) where we already know the sitemap location. This could recover some latency for common cases.

// Potential optimization
if (library in KNOWN_DOC_STRUCTURES) {
  // Skip websearch, go directly to sitemap
  webfetch(KNOWN_DOC_STRUCTURES[library].sitemap)
}

But this is a nice-to-have, not a blocker.

LGTM - The precision improvement justifies the latency cost.

GMGM any updates here?